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AB STB ACT . • . 

Teaching may seea tec ccaplex and too involved in 
various diaexisions of affective and cognitive aspects of learning fot 
a good tea'bher evaluation prcgras to te devised* A hierarchy of 
criteria is proposed, based upon analysis of obser^vable versus 
unobservable characteristics, neasureafcle and ncn-ieasureable 
characteristics, and attributes and variables* Cnce reliability is 
established, then the conceptually acre difficult task of deteraining 
validity can be^in. The ultimate goal of validity can only fce based . 
upon reliability* The researich task of developing rel^iable aethods of- 
evaluating teachers and deteraining the validity of the criteria on* 
dhich these aethods rest lies before us* (Authox/C^H) 
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"One the issues in teacher evaluation tha^ needs 
to be resolved; possibly before all' others, is that of 
determininq the .criteria for measiJrina teacher effec- ' 
tiveness. The criteria depend on a hiqh degree of 
validity to be useful to a teacher evaluation program. 
This article attempts to develop a ^foundation prereq-- 
uisite to the ^xamTilation of the validity of evaluation 
criteria and to indicate direction for further res^earch 
on teacher evaluation. 



Method of Review 



» . \ 

A review of the research literature on teacher 

evaluation criteria was made with reference to the 

Research itf Education (ERIC) - January, 1972 to December^ 

1974 and the Resources in Hlducation (ERIC)- ^- Januarv, 

1975 to November, 1976." 'Simon and Boyer's Kirrors for 
Behavior: An Anthology of Classroom Observati^;^ (1967) 
and Gage's Handbook of Research on Teaching (1963) were 
widely used as* additional references. For an overall ^ . 
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survey /jf tKe educa'tprs ' efforts on^teacher evaluation , 

Barr's Wis*c;;gnsin Studies of the Measurement ^nd Predic - 

tion of^-Teacher Effectiveness (1961) and Biddle and 

Eliena's Contemporary Research ^on Teacher- Effectiveness 

* « 

(1964) were freq|iently referred to. In addition, 
DubiiJi's Theory Building (1969) was used as a basis for 
st/ucturing the theoretical background of -this ^article.- 

^ Findj.ngs 

I . • ' ' 

In the Preface to Contemporary Research on Teacher 

Effectiveness , Biddle and Ellena (1964) point out that: 

, . . with all this research activity, results 

have been modest and often cor^radictory • 

Few, if any, facts are .nov^ deemed established 

about teacher effectiveness; and many ^former 

findings' have bi&en repudiated.. It is not an 

exaageration to say £hat we 'do not today 

know how tO' select, trarin- for, encourage or 

evaluate teacher effectiveness (Preface^, p. vi) , 

In the Evaluation of Teacher Effectiveness , 

McDaniel (1972), after reviewing Biddle and Ellena^'s 

remark, says: . 

Considering the present state of artV« the 

temptation is large for becomino a prophet 

of doom crying out against further efforts 

• t6 undertake so fruitless a task as teacher 
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evaluatioh (p . 1) 



\ Even • though the process of teacher evaluation as 
described by these educators is very complicated, it i's" 
certainly not a de^d-^end process. In fact, the effec- , 
tivenes's of the evaluation program depends to a great | 
exteht on the validity of the criteria ' used in the 
evaluation of teachers., The^fore, program developers, 
must decide what the teacj^^rs are to be evaluated upon; 
in other words, *what^^e evaluation criteria '^re to be. 
In addition, they n>dst be sure that trie criteria bhey # 
choose are valid Only^ valid criteria can .result in 
meaningful teacher evaluation. 
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* The ^Gbod" Teacher 

Educators have 'identified the characteristics or 
behaviors of good .teachers {e%g. , A.S. Barr, 1958 

and D. Hamacheck, 1968).^ 'When these characteristics 

< 

•or behaviors are perceived f jrom an evaluation point 
'of vi^w, they can be^ divided into observables and 
unobservables with the observables further * subdivided 
into attributes and variables, as well^ as measurables 
and -non-measurables (see: Diagram 1) . 



Characteristics or behaviors 
of a good teacher 



Diagram 1 



The observables 



The unobservables 




The hon-measurables 



The variables 




The attribute^ 
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The best eval 



nation program that ca^i be devised^ is 



np matter how care 
teacher characteri 



^^^4. 



limited to observable characteristics of behaviors^ecause 



jfully we observe*, there are still good 
^tics or behaviors we cannot observe. 



or ^ 



W€i can identify a |ist of characteiiistics or behaviors 
of good teachers, l)ut we cannot necessarily expect a 
person wh6 posse^ss^s these characteristics or behaviors 
'to be a good teacher. As Dubin (1969) 'says: " 
^ * We simply arre not capable o£ seeingV things 
whole . Nor is man capable of retainih-g^-a^^d 
recording cort\plex phenomena coming within the • 
range of his sensory fields. It is necessary 
to acknowledge that man, who builds theories 
to model his world of observation, has genuine 
limits on his capacities to grasp complex 
observations (p. 30) . 

The meaqurables are, those characteristics or • 
behaviors that Can be measured by special ^devises in 
relation to criteria. The non-measurables are those 
that cannot be measured by any device or in any other 
form except'' as subjective opinioiy. Many educators 
have been interested in the measiurability of teacher 
characteristics. Wallen and Trovers (1963) ooint out 
that "evan %Tl thc|se situations defined strictly ^n 
'terms 'of "^ob^ervablea, such observables are ' themselves 
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abstracticxns* : ♦ " (p, '448) Redfern- (1963) refers to a 
teacher's personality traits, habits, manner, judgment, 
'emotional s.tability, initiative, tact afid courtesy as 
being hard to evaluate. , Gibson and Hunt (1965) ^also 
state that educational outcomes are often' hard to 
measure, ' Stoops, Rafferty and Johnson (19'75) talk about 
pupil subject achievement being measurable while, social , 
moral,. and emotional progress are relatively intangible. 
On the whole , qood teacher characteristics or behaviors 
contain abstractions which are non-measurable. 

In defining an attribute and a variable, DOfbin 
(1969) states: 

^ An attribute is a property of a thing dis- 
^ tinguished by the quality of being present. 

The thijig always has this quality if the 
a'ttribute -is a property of the thing,' All 
things having a qiven attribute property 
constitute a set of identities on that 
attribute property. All other things are 
in a ^et* -identified by the Lack of the given 
attribute property, « A variable is a property 
of a thing that may be present in degree. 
There may be some of the property present 
or a lot of it. , , What is significant 
when we employ a variable unit in a theory 
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is that^ our attention becomes focused upon the. 

afmount or deqrees to which this property is 

pre^^rii^ in th^ thing (p, 35), • ' 
y In other words, by attributes, we mean the gener- 
ally common' characteristics or behaviors that can be 
identified as' existing in gO|bd' teachers. By variable^ 
we mean the quantity of chai^acteriistics or behaviors 
that are rec^uired of individual teachers. In the 
^ense of essentiality, attributes constitute an inevi- 
table part of the characteristics or behaviors of good' 
teachers, while variables are present in different 
degrees of importance, 
Egtablishinq -Reliability 

Establishing teacher evaluatior^ criteri^ b^sed 
upo'n^ good teacher characteristics or behaviors is a 
common practice "Of the school administrators. However, 
the.; question of the validity of these characteristics 
or behaviors remains unanswered. 

To examine tl^e .validity of teacher evaluation 
♦criteria, an attefnot is n3ade to design a hierarchy of 
reliability as follows: 

If the measurable attributes are represented by 
A, the measurable .variables by B, the non-jneasurable 
attributes by C, and the non-measurable variables by 
D; and vi*f it is assumed that as evaluation criteria. 



the attributes are more reliably-observed than the 
variables and the measurables are more reliably-observed 
than, the non-measurables, 'then A, B, 'C and D can be 
arranj^^in a simple hi,er:archy of reliability (see: 
Diagram 2) for examining Jbeacher avaluation criteria 
with D, C, B and A in an -ascending order of reliability. 



Diagram 2 



Reliable 









A X 






B / 






c 







Unreliable 
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In evaluation programs^, the criteria are often 
based on more than <5ne t^pe of teacher characteristics 
or behaviors. Therefore, there are possibilities of 
two* or three or even four types of teacher character- 
istics or behaviors involved. In considerino the 
involvement of only two types of teacher characteria- . 
ticg or behaviors, a more complicated hierarchy of 
reliability can be constructed- through pairings (see: 
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Diagram 3)''with *( C + D) , (b'+.D), (B + C) , (A + D) , 
(A C) and (A. + B) "in ascending order of reliability, 



Reliable 



Diagram 3 









/) -t- a / 
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B + D . 






G 4- 


D ^ 







tinfeliable 



In the saiDe way, in the involvement of three types 



of teacher characteristics or behaviors, anotjier more 

complicated hierarchy of reliability can be .constructed 

throuqh triads (see: Diaaram 4) with ;{B + C + D) , 

"(A + C + D)., (A + B 4- D)' and (A + 'B + C) in ascending' - 
order of reliability. 



Diagram 4 



Reliable 















6 -+-C -f-a 





Unreliable 
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'Moreover, tlie utilization of four tj^pes of * teacher 

acteristics or behaviors can be expressed as 

* 

. » 

(A + B + C + D)^ . " . ' 

By using the same previous assumptions , considering 
the possibility of A + B + c. ^ D, and farther assuming 
tha't the components in the pairs, triads and • the y 
quartet are proportionally ^qual in quantity {e..g,, 
A + B means A and B^are proportionally equal in quantity) 
then diagrams 2, 3 and ^ can be logically combined to 

i 

derive a more 'complete fifteen level hierarchy of 
reliability (see: Diagram 5) • \ ' . 
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The simple hierarchy^ -in* Diagram'"-'2 can be' used to. 
examine the reliability of individual teacher evaluation 
criterion. The complicated hierarchies in Diagram 3 and 
4 can f'be employed as a means' ta , compare the degree of 



reliability . among ^evaluation pi?ograms^ io^lving "pairs" 
and "triads" characteristics or behaviors. As, tp^the* 
fifteen level hierarchy in Diagram 5, it isM^^^B^o 

examine the 'reliability of any teacher evaluation , pj^o- 

gram. . • ... - 

» 

Needed Research 
More ^ese_arch is needled to clear up maijy unsolved 
oroblems relative to the reliable* feeacher evaluation 



methods. Some of the pressina problems are given 
below : ' - ^ * . . ' ' 

In constructing the hierarchy of reliability 
(Diagram 5) , it is Assumed tha"C^the.sattributes and the 
variables can be identified. But, in aqtual practice , 
to distinguish attributes from variables involves . 
much personal judgment. However, since "an attribute 

is a property pf^a^ thing distinguished by the qualiVy 

* * * 

'of being present and al^ other things are i*n a set 

identified by the lacjc of the givejn property,'* tile 

attributes -and variables can be described. In Diagram 

6k. the three circles stand for three different evalu- 
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ai^ion ^prograips , the shaded area represents the attribute 
cri'te'^ia/ ^ancr the "unshaded areas, the variable criteria. 




In other words, criteria th'at' are conponly u^ed 
in many evaluation -programs are attribute criteria and 
criteria that are ndt commonly used in evalua"t;ion pro- 
gifams are variable criteiTf'a, As long as it is diffi-' 
cult 'to determine the ,dea^?fee *of commonness, the problem 
of attributes and variables leave much to Ke researched. 

Through years of research, educators keep admitting 
that* there are teacher characteristics or behaviors 
tljat cannot be measured • But intellectual advancements 
have been able to make many things possible. Concerned 
educators have never ceased* their efforts in attempting 
to experiment with new strategies of evaluation. There- 
fore, evaluation program planners should be aware of the 
fact that regardless of the new developments making many 
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teacher characteristics or behaviors n>easurable, there 

' t 

ds ' still much needecl research 'in the area of J:he non- 
measttrables . 

It is assumed in Diagram .S that the component 



\ — ^ -^v^ . 

criteria in th6 pairs, the triads and ,the quartet. 

/ 

(e^q.,- A + B, A + B + and A + B + d D) are 
proportionally equal in quafit'jtty. But in actual 
practice, the component criteria are frequently not 
quantitatively *equal in proportion, Appropriate adjust 
ments are required. There is a need for studies of the 
degree of appropriate adjustments . • ^ 

Conclusion 

Teaching may seem too complex and too invo-lve.d in 
various dimensions of affective and' cognitive aspects 
of learning to devise an -universal good teacher evalu- 
ation program. Howeveir, teaching can be evaluated in 
a meaningful sense if, and 'only if, we can establish 

reliability in our methods. Although the hierarchy .of 

» * 

• e 

reliability (Diagram 5) is not a technically satisfac- 
tory device, it is expecl^ed that the idea c^n suggest 
other alternative directic5ns to a more practical 
strategy of teacher evaluation. » 

Once reliability is established then the conceptu- 
ally more diffiqult task of deterijiining validity can 
begin. The ultimate goal of validity can oniy^ be based 
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upon ^reliability . The research task of- developing 
reliable-^ methods of 'evaluatinq^eachers and de*termining 
the validity of the criteria on which these methods 
rest lies before us . . • 
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